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ABSTRACT 

A project was undertaken to develop a guide to help 
small business programs select good education and training programs 
for their staff. An initial step was a review of literature related 
to adult education and training in small business and training 
evaluation. The initial plan was to have the guidebook be specific to 
a given industry or other appropriate group. Four different criteria 
and three different ways of choosing a critical category of program 
were considered. However, the advisory committee decided upon a 
broader guidebook that would: help small businesses select a training 
program from outside vendors; not be specific to a particular type of 
training program; and not be limited to certain types of providers. 
Training and education programs offered by large national companies, 
small local vendors, community colleges, and training consortia were 
analyzed, and focus groups were conducted with small business 
personnel with respect to their current training practices. The 
guidebook that was developed consisted of three sections: training 
needs analysis; guidelines for selection; and evaluation. Reaction 
from a field test showed that some liked it very much and felt it 
would be very useful, whereas others felt it was too simplistic. The 
guidebook was determined to be useful for the small business person 
who was not a training professional at a firm that was actually 
seeking training for its staff. Dissemination options were 
investigated. (A program evaluation checklist is appended. Contains 
52 references.) (YLB) 
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Chapter 1. Introduction 



The purpose of this contract was to develop a guide to help sniall business 
programs select good education and training programs for their staff. A guidebook has 
been developed that is ready for publication and distribution to the target audience. The 
steps taken in fulfilling this purpose were not always exactly the steps laid out in the 
original proposal. The original plan was modified based on information collected by the 
American InsHtutes for Research and input from the Advisory Committee. In this final 
report we will describe the work that was actually completed in developing the 
guidebook and discuss some of our conclusions and recommendations for further work 
in this area. 

The project Advisory Committee consisted of both experts in the education and 
small business fields, as well as represmtatives who were working adults. The 
committee had three meeHngs and guided the project at every stage. The members were: 

Gary Andrews, small business representative 
Human resource director with Pulsecom Hubbell, Inc., Herndon, VA 

Laurie Bassi, Ph.D., labor economist 
Professor in the Department of Economics, Georgetown University 

Miriam Burt, union representative , „, . . . r^r- 

Analyst with the Food & Beverage Workers Union Local 32, Washmgton, D.C. 

John Henderson, working adult and small business representaHve 
Machinist with Garland Laboratories, Silver Spring, MD 

Philip A. Jury, Ph.D., organizational development specialist 
President of SPR, Minneapolis, MN 
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Stuart Rosenfeld, Ph.D., small business expert 
President of Regional Technology Strategies, Inc., Chapel HiH, NC 

Anthony Sarmiento, union representative 
Manager with HRDI, AFL-CIO, Washington, DC 20006 

Gail Spangenberg, workplace literacy expert v u xtv 

Vice President of the Business Council for Effective Literacy, Inc., New York, NY 

Lilburn Williams, small business representative 
President of Williams Associates 

Susan Zapolskj, working adult 
Research Analyst, U.S. Postal Service, Washington, D.C. 

The original proposal called for six tasks to be completed: 

1. Form an Advisory Committee 

2. Review the literature 

3. Develop a framework 

4. Collect data 

5. Develop and validate the model 

6. Prepare guidelines 

We began by completing the first three tasks as planned. However, the framework we 
developed was much broader that originally envisioned. The initial plan was to have the 
guidebook be specific to a given industry or other appropriate group. The Advisory 
Committee felt, and the Department of Education was convinced, that a broader 
guidebook would be best. We then collected data from different kinds of training 
sources and conducted focus groups with small businesses in preparation for developing 
the guidebook The guidebook went through several revisions and was finally tested m 
the field with some small business personnel. The final guidebook has been submitted to 
the Department of Education. 

The report is organized into seven chapters. The next chapter discusses 
background and the findings from the literature review. We have included some works 
idenHfied after submitHng the literature review. Chapter 3 presents our considerations m 
choosing a framework for the project. Data collection is described in Chapter 4, both m 
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terms of our study of different training alternatives and the focus group results. Chapt. 
5 discusses the results of our usability testing. We discuss our conclusions and 
recommendations in Chapter 6 and dissemination options in Chapter 7. References are 
included at the end of the report The Program Evaluation Checklist is in the Appendi: 



TJte American Institutes for Research 



7 



3 



Chapter 2. Background and Literature Review 

One of the initial steps in this project was to review the literature related to adult 
education and training in small business and training evaluation. In this chapter we will 
include some of the sources we uncovered during the initial search, as well as other 
sources updating the search from that time. 

We classified the materials into one of four categories which guided our search. 
A. Training in Small Businesses 

In developing a guidebook, to be used primarily by small businesses we wanted to 
gather the most recent information on how such firms currently train their employees. 
We were able to locate a number of books and articles that speak to this point. 

Small businesses have generated many of the jobs created in recent years. They 
also are more likely to be employers of entry-level workers (Business Council for 
Effective Literacy, 1990; U.S. Small Business Administration, 1988; Office of Technology 
Assessment, 1990). As entry level employees, these workers are frequently in need of 
training. Over the next ten years there will be a shortage of such workers and it will be 
necessary for these firms to employ more minorities, women, and immigrants, who may 
have even more of a need for training than current employees (Business Council for 
Effective Literacy, 1990; U.S. Small Business Administration, 1988; Lichtenstein, 1989; 
Szabo, 1991). 

Small firms are not currently in a good position to take on this challenge. They 
offer less training in every category than large firms (Business Council for Effective 
Literacy, 1990; U.S. Small Business Administration, 1988; Banks et.al., 1987; Lee, 1991). 

One of the reasons could be higher turnover at smaller firms (Bishop, 1991a; 
WAVE, 1991). Also, training that is not specific to the firm may be lost to the firm if an 
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employee leaves to work elsewhere (Bishop, 1991b). When considering training, small 
businesses are very concerned about costs and also Hme away from the job (Business 
Council for Effective Literacy, 1990; Banks et. al., 1987; Kelly and Thompson, 1988). 
Persons in small firms are needed more urgently and it is harder for them to be released 
from work to attend training. 

Another reason for the lack of training in small firms is that frequently there is 
nobody employed by the small firm to handle training and employee development 
(Banks et al., 1987; Kelly and Thompson, 1988). Instead, the president, or perhaps an 
administrative assistant, takes on this role. Therefore, the person selecHng training for a 
small business usually has no expertise in education and training. 

Small businesses are more likely to use trade associations, college courses and 
technical training sources than large businesses (Banks et. al., 1987; Lichtenstein, 1989). 
Lee (1991) found that small businesses appear to want the same kinds of courses as large 
businesses and basic skills training is on the bottom of their list. Nevertheless, this may 
be the type of training most urgently needed (Lichtenstein, 1989; Office of Technology 
Assessment, 1990). Small firms who have promoted basic skills training appear to have 
received benefits to their organizations (Szabo, 1990). Firms who continue to support 
workplace educaHon programs feel they are worthwhile in reducing error and 
contributing to employee morale (Bassi, 1992). 

These findings influenced our guidebook in two ways. Our guidebook had to be 
educational, since there are usually no training and development personnel in small 
businesses. We had to write the guidebook so that a person without any training 
background could understand it. Second, the difficulties for a small business in finding 
affordable training offered at a convenient time and place should not be minimized. We 
wanted to provide some practical guidance as to where to look. 

B. Training Evaluation Models 

Brandenburg and Smith (1986) provide a good overview of the history of training 
evaluation studies over the past thirty years. We concentrated our search on methods of 
summative evaluation, which is concerned with program effectiveness, rather than 
formative evaluation, which is concerned with testing training results as a course is being 
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developed. A discussion of formative and summative evaluaHon can be found in Gagne 
et al. (1988). 

Kirkpatrick (1967) iniHally formulated a model for training evaluation which has 
been extremely popular and is still used in many evaluations, both in the United States 
and overseas. The four-level model looks at reaction, learning, behavior and results. 
More recent approaches frequently are just a variation of this basic model. For example, 
the American Society for Training and Development Guide (1986) has five evaluation 
criteria that could be seen as Kirkpatrick's second level divided into two categories. 

More recently there are a number of new trends in evaluation. Brinkerhoff (1987) 
has developed a six stage model of evaluation which differs from Kirkpatrick's approach. 
His point of view is more ^riented to the business perspective and de-emphasizes the 
reactions of trainees. Other models also oriented toward the business perspective that 
have appeared in the literature are cost-benefit models such as Basarab (1990), which 
provides detailed information on how to calculate a return on training investment IBM 
uses an input-process-output model of evaluation (Bushnell, 1990). Fairfield-Sonn (1987) 
presents a strategic process model that he specifically orient^ toward small business and 
provides an example of its use. 

One of the most recent books on evaluation implicitly criticizes the Kirkpatrick 
approach in that it reports findings that trainee reactions appear to have no relationship 
to how much trainees learned or retained pixon, 1990). She argues that training 
evaluaHon must not be based on employee reaction, but rather on measures of 
performance and on-the-job behavior. Some previous users of Kirkpatrick's levels of 
evaluaHon have argued that they are dependent upon one another. For example, if 
trainees have a poor reacHon to the instructor they are less likely to learn. If the levels 
are not related to each other empirically, then knowing student reactions will not tell us 
anything about learning, behavior or results. 

Although most of the training evaluaHon literature stresses a quantitative 
approach, there is a school of thought that sometimes qualitative evaluations are more 
appropriate. Representative of this point of view is the Patton (1980) book. 

Some of the evaluation criteria which would provide the best information for 
decision-making are the most difficult to collect. Calculating the return on investment 
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may be possible for IBM, but is it practical for a small business? If only one employee is 
being trained, making the calculation could be time-consuming with little potential 
payoff. On th i other hand, if a firm bases its decision to choose a certain vendor based 
on past participant reaction, will this necessarily mean that this program will be effective 
for the new firm? 

C. Training Evaluation Studies 

It is difficult to locate training evaluation studies, particularly those that evaluate 
package programs. This appears to be primarily due to the fact that many such studies 
are in-house products that are produced as an internal report, but never published in the 
more formal literature. There have been several large surveys of businesses concerning 
their training evaluation practices, which are discussed below. In addition, there have 
been a few surveys of training evaluations, some of which have relied upon published 
studies and some of which have relied upon unpublished studies. We have also 
included recently completed evaluation studies that we have been able to obtain in 
report form. 

A series of articles have surveyed businesses to find out how they evaluate 
training programs (Campbell et. al., 1970; Catalanello and Kirkpatrick, 1968; Clegg, 1987; 
Delaney et. al., 1989; Kirkpatrick, 1978). Despite the wide range in publication dates, a 
major conclusion remains the same: Most firms evaluate courses based upon trainee 
reactions. In addition, it appears that most firms do not feel this is the most effective 
method. Grider et. al. (1990) found that the methods rated by human resource 
development professionals as most effective were: behavior evaluation, competency- 
based evaluation, and correlation evaluation. y 

Another group of articles analyzes actual evaluation studies rather than business 
practices reported on surveys (Brandenburg, 1989; Dubin eL al., 1974; Hawthorne, 1987). 
Once again a major conclusion is that the most common criterion for evaluation is 
student reaction. The second most common criterion is measures of improvements of 
skill or knowledge. 

We have been able to find few recent training evaluation studies (Abt Associates, 
1990; Creticos and SheeLs 1989; Kutner et.al., 1991; Manpower Demonstration Research 



Vte American Institutes for Research 



11 



Background and Literature Review 



Corporation, 1988; Meridian Corporation, 1991a, 1991b, 1991c; Nightingale et. al., 1991; 
Yin, 1989). Those that we did find confirmed the pattern discussed above. Kutner et al. 
(1991), for example, reports that assessment of the program usually relied upon anecdotal 
evidence by instructors and participants. 

Although everyone agrees that it is very important to evaluate training, it appears 
that this is rarely done. Even when there is a training evaluaHon most of the studies are 
not published so it is difficult to locate them. In choosing a training program we 
recommend asking the vendor about previous evaluations of the course, but most 
programs will probably not be able to satisfy this -equesL 

D. Guides to Choosing a Training Program 

The need for a guidebook for selecting training programs is underscored by the 
observation in Carnevale et al. (1990) that most training in the U.S. (up to 90 percent) is 
based on off-the-shelf products. With a wide selection of potential vendors for any given 
course, developing a good method for evaluating them is very important. Previous 
articles and brochures present criteria for vendor selection that we considered in 
developing ours. 

Many of the guides mention that tho organization must compare their training 
needs with the objectives of the program (American Society for Training and 
Development, 1988; Martin, 1983; Rogers and Volpe, 1984; Scriven, 1974; Sticht, 1991). 
We included a section in the guidebook about training needs analysis. 

Certain criteria were found in many of the references listed, including: 

Cost or cost effectiveness (American Society for Training and Development, 1988; 
Carnevale et. al., 1990; Rogers and Volpe, 1984; Scriven, 1974; Sticht, 1991; 
Tartell, 1987) 

Evaluation Results (American Society for Training and Development, 1988; Martin, 
Phillips, 1983; Scriven, 1974; Sticht, 1991; Tartell, 1987) 
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Delivery Method (American Society for Training and Development, 1988; 
Carnevale et. al., 1990; Martin, 1983; Phillips, 1983; Tartell, 1987) 

Expertise and/or Experience of Vendor (American Society for Training and 
Development, 1988; Carnevale et al., 1990; Phillips, 1983; Sticht, 1991) 

Content (American Society for Training and Development, 1988; Carnevale et. al., 
1990; Martin, 1983; Phillips, 1983) 

References (American Society for Training and Development, 1988; Phillips, 1983; 
Tartell, 1987) 

Materials (American Society for Training and Development, 1988; Tartell, 1987) 
Support (Carnevale et. al. 1990; Scriven, 1974) 
Environment (Martin, 1983; Phillips, 1983) 

It is interesting that evaluation results are considered important by most authors 
who have developed selection criteria. The question is where are those evaluation results 
coming from. If they are from the vendor, there is the obvious problem of objectivity. 

There were other criteria that were only mentioned in one article or book: 

Availability (American Society for Training and Development, 1988; 

Philosophy (Carnevale et ah, 1990) 

Pilot testing (Rogers and Volpe, 1984) 

Many of these criteria mentioned by others were appropriate to include in the 
guidebook. After completing the research described in Chapter 4, we developed our 
seven guidelines that were used in the guidebook. 
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Guidebook 



The original research proposal planned to develop a guidebook, for some 
particular subset of small businesses that would enable them to choose effective training 
programs for their employees. We planned to choose a "critical" category of programs 
and then develop a guidebook, for that category. In this chapter we will describe our 
considerations in choosing a category. The Advisory Committee recommended against 
this approach and the framework we eventually adopted will be presented at the end of 
the chapter. 

A. Criteria for Choosing a Critical Category 

We considered four different criteria in trying to choose a critical category. 

• Potential Use 

One criterion is the number of businesses, both small and large, that could make 
use of a guidebook concerning a chosen category. For example, if the guidebook, 
concerned training programs for a rare and unusual occupation that existed in 
only 75 businesses in the United States, it would have limited circulation. We 
wanted to have a guidebook that could be of use to a wide audience of businesses 
and would reach a large number of individuals who need training. 

• Current Relevance to National Needs 

Many Americans are concerned about the state of U.S. business and the economy. 
Consensus seems to be building that the education and training of the labor force 
is critical to the nation's prosperity. One criterion for the guidebook is to have it 
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address a segment of training where there is currently much need. An example 
would be training that addresses the shortage of entry-level workers in the 1990s. 

Future Relevance to National Needs 

Instead of focusing on the current situation, another approach is to look to the 
future. What kind of training might be needed for preparing the workforce for 
future jobs? A guidebook that enabled business to better prepare workers for 
high technology jobs in the future might be very useful. 

Small Business Needs 

Based on the small number of studies of small businesses that we could find, 
small businesses appear to feel that certain types of training are more important 
than others. We could choose a category that small businesses would be more 
likely to use because it would meet their perceived needs. 

B. Possible Critical Categories 

We considered choosing a critical category in three different ways: by type of 
program, type of provider, and type of trainee. Within each of these categories we 
considered several alternatives. 

Type of Program 

Developing a guidebook that focuses on a particular type of training program is 
intuitively appealing. Once a category was chosen and further defined, it would be easy 
to idenHfy relevant training programs for study. Three types of programs we considered 
were: basic skills, computer skills, and management training. 

Currently there is considerable interest in basic skills and workplace literacy. The 
concern is that many American workers do not possess the basic competencies to 
function in the workplace. Despite the discussion in the literature of how important 
basic skills are in small business, this may not be the common perception in the firms 
themselves. Basic skills was on the bottom of the list of the types of training currently 
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provided in a survey of training practices in small and large businesses (Lee, 1991), and 
may not be the best choice for this reason. 

Another possible category we considered was computer skills. Such skills are 
becoming a larger and larger component of people's jobs. In 1988, 70 percent of fixed- 
capital investment was computers and telecommunications equipment (Coates et. al., 
1991: 16). Computer literacy is even more important in the future for small businesses. 
The growth in manufacturing firms employing fewer than 100 workers was strong in the 
latter half of the 1980s due in part to "use of computerized machine control technology 
that made it feasible to turn out goods in shorter production runs" (Business Council for 
Effective Literacy, 1990: 5). 

A third possible type of training we considered was managv^ment training. On 
the top of the list of training that small employers offer, or want to offer, is management 
training (Banks et. al., 1987; Lee, 1991). Both small and large firms give management 
skills the highest rank. If the guidebook centered on this area it would presumably be 
useful for all of the companies that send employees to such programs now. However, 
this would not focus on the problem of the entry-level workers. Those who get 
management training tend to be more highly educated anyway. There is growing 
concern that it is the less-educated half of the workforce that needs more attention. 

Type of Provider 

Another way of choosing a critical category is to focus on a particular type of 
training provider. The guidebook would then be relevant for companies who are 
considering sending their employees to that type of program and would help them to 
decide if a given type of provider were a good choice. There are many possibilities for 
categorizing training providers. We considered community colleges, training consortia, 
and specialized training companies. Junior and community colleges have become 
important providers of education and training to a variety of workers. One-third of the 
providers of cuteide training in the study of 18 small business workplace learning 
programs by Berkeley Planning Associates (Venciil et. al., 1991) were community or 
junior college staff. Training consortia are considered as a possible solution to the 
problem of small businesses meeting their training needs in the Office of Technology 
Assessment (1990) report on worker training. Within a given geographic location or 
within a particular business market, such as banking, small businesses form a consortium 
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to offer programs to members of their group. In this way small businesses can 
potentially gain some of the economies of scale available to larger firms (Lichtenstein, 
1989). Unlike larger firms, which have training departments that develop in-house 
programs, small businesses are heavily reliant upon outside firms. It is difficult for a 
small company to differentiate between marketing and substance. 

Type of Trainee 

Yet another way of choosing a critical category is to focus on the type of trainee. 
In this case the guidebook would be specifically focused on meeUng the needs of certain 
types of employees. We considered five possible categories: entry-level workers, 
immigrants, older workers, high school dropouts, and displaced workers. 

Choosing entry-level workers as a category would be similar in some ways to 
choosing basic skills. A small business might find it easier to identify entry-level 
workers and then use the guidebook, compared with having to first decide if its workers 
are deficient in basic skills. 

It is expected that some of the shortfall in the number of young workers entering 
the labor force in the 1990s will be made up from the immigrant pool (Lichtenstein, 1989; 
U.S. Department of Labor and U.S. Department of Education, 1988). Immigrants have 
special needs, not just in terms of language, but also in terms of understanding and 
adapting to American culture. 

Another possible category is that of older workers reentering the workforce. As 
already stated there is a need to make up the shortfall of younger workers from another 
source. Women who have taken time to raise a family may enter the workforce at an 
older age. Older workers have a different perspective and needs compared with 
younger workers. They may have stronger moHvation and problem-solving skills, for 
example. However, in other areas they may need more help. An example would be the 
older person who is afraid of computers compared with the younger person who has 
used them in school. 

At a conference held by the National Alliance of Business, Thurow (National 
Alliance of Business, 1991) emphasized the importance of educaUng the bottom half of 
the workforce. Much of the educational system focuses on those students who are going 
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to college. Students who do not even graduate from high school cannot find jobs easily 
due to lack of skills. 

In her book on management challenges for the 1990s, Kanter (1989) discusses the 
increased mobility of the workforce. The pattern of staying with one organization for an 
entire career is disappearing. People change jobs many times during their working lives. 
Although professionals with strong educational backgrounds can make these transitions 
on their own, many lower level workers may need more help. Having mastered certain 
skills in a particular industry, they may need to be extensively retrained in another 
occupation. 

C. The Chosen Framework 

At the first meeting of the Advisory Committee, the possible categories that could 
be used for the guidebook were presented. The consensus of the committee was that the 
guidebook should be aimed at the broad spectrum of small businesses rather than 
restricted to a particular sector. None of the categories described in the first part of this 
chapter were adopted. However, we did define an audience for the guidebook and a 
framework for collecting data on training programs. 

The audience for the guidebook is small businesses, defined as those having 500 
employees or less. Within this definition are a wide range of businesses. We decided to 
focus particularly on companies at the lower end of the range. These are the companies 
who are not likely to have a training professional on staff. 

The guidebook is intended to help small businesses select a training program from 
an outside vendor. We did not concern ourselves with on-the-job training or career 
development. If a small business has defined a training need, the guidebook helps them 
to evaluate the alternatives. 

The guidebook is not specific to a particular type of training program, such as 
basic skills. Rather it is aimed at helping to identify and examine existing programs that 
could be focused on any workplace training need. When we collected data on training 
programs we did limit our search to certain types of programs. However, the guidebook 
is not restricted to certain programs. 
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We also did not limit the guidebook to certain types of providers of training. In 
fact, the guidebook discusses the pros and cons of various kinds of training providers, 
including: community colleges, national training developers, local training consultants 
and community business associations. 
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Chapter 4. Development of the Guidebook 



In order to develop a guidebook that was realistic for the small business user we 
collected information from two sources. First, we studied training and education 
programs offered by large national companies, small local vendors, community colleges 
and training consortia. We wanted to see the kinds of information that would be 
available for a company seeking training. Second, we conducted focus groups with small 
business personnel to gain a better understanding of how they view training so that the 
guidebook could be structured to better serve their needs. 

A. Analysis of Training Programs 

Training and educaHon programs are available from a wide variety of sources. 
The issue is what kinds of information can be collected on a given program that might 
aid in selecting one that will best suit a given training need. We contacted vendors from 
four different sources of training and asked them about given courses. The most time 
was spent with large training companies, but we also surveyed small local training 
companies, training consortia and community colleges. 

We developed a selected list of types of training programs-for example, word 
processing~to guide our data collecHon. This list was intended to be the most common 
types of training that small businesses might select. We used the training survey that 
appears annually in Training magazine to find the programs to include. This survey did 
not include companies with less than 100 employees, but did have some data shown 
separately for companies having between 100 and 499 employees. We included a 
program if: 

60 percent or more of businesses between 100 and 499 employees provide 
this type of program 

• 40 percent of all companies provide this type of training 
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30 percent of all companies use outside vendors, either partially or 
exclusively, to provide this training 

To aid in collecting information we developed a Program Evaluation Checldist 
which lists various instructional characteristics that past research tells us lead to effective 
instruction. A copy of this checklist is included in the Appendix. 

Large Nationwide Training Companies 

We attended the 1992 meeting of the American Society for Training and 
Development in order to talk with training vendors. Our procedure was to first 
approach a vendor to obtain brochures on the program of interest. After read mg the 
brochure we returned to the vendor to discuss the course in more detail. If possible, we 
examined course materials. In a few cases only we were about to observe a course. The 
number of programs studied is shown below in Table 1. 

Table 1 

Number of Each Type of Training Program Offered by Large Training 
Companies Included in the Sample 

Number of Companies 



Management Skills/Development 
Delegation Skills 
Decision-making 

Managing Change 
Supervisory Skills 

Motivation 

Problem Solving 

Negotiating Skills 
Communication Skills 

Interpersonal Skills 

Team Building 

Listening Skills 

Writing Skills 



1 
2 
2 

1 
2 
3 

1 
5 
2 
7 
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Number of Companies 



Basic Computer Skills 1 

Word Processing 2 

Personal Computer Applications 2 

Computer Programming 2 

Data Processing' ' 2 



Management Information 
Systems ^ 
Quality Improvement 
Customer Relations /Services 2 
Executive Development 1 

Leadership 3 
Clerical /Secretarial Skills 
Personal Growth 

Time Management 3 

Goal Setting 

Stress Management 1 
Total Number of Programs 



In about half of the cases we were able to obtain a brochure and also interview 
some of the sales personnel about the course (Table 2). For another third of the 
programs we were also able to examine materials. Obtaining this informadon took 
persistence, however. Training vendors do not appear to be accustomed to having a lot 
of quesHons asked about their programs. The persons who work the booths at the large 
training conventions are usually sales personnel, rather than instructors or course 
developers. 
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Table 2 

Sources of Infonnation Collected About Programs Offered 
by Large Training Companies Included in the Sample 

Number of Programs 

Brochu. Only ^ 
Interview Only ^ 
Brochure and Interview 24 
Brochure, Interview and Observation 1 
Brochure, Interview and Materials 16 
Brochure, Interview, Materials and Observation 1 

Total 46 

By using all of the possible sources of informaHon, we could sometimes obtain the 
information needed for completing the program evaluation checklist and someHmes not. 
Table 3 shows the percentage of Hme that the information was available. InformaHon on 
course content was relatively easy to obtain. For all items, we were able to obtain the 
information from at least 85 percent of the programs. InformaHon about the insh-uctor 
was more difficult. We did not include items about the instructor in the final guidebook 
we developed. Past experience of the firm was mixed. Most companies could not 
provide informaHon on their past experience with small business firms, because they do 
not keep their records in this manner. InformaHon on cost and availability could usually 
be obtained. The last item concerning course effectiveness was information about any 
past evaluations of their programs. About half the firms could provide this. 
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Table 3 

Percentage of Companies Supplying Information Needed for 
Course Selection and Evaluation 



Procram Evaluation Checklist Items 


Percentace of 
Companies 
Supplying 
inrurinuiiwii i 


L Needs Analysis 
Will do TNA 


100% 


IL Course Content 




Objectives available 


98 


Outline available 


96 


Pretest of competencies given 


91 


Postest of competencies given 


87 


Adjustments made based on course 
participants 


96 


Methods used 


QQ 1 


Appropriateness of methods 




Active learning opportunities 




rractice oi new skjus 


91 


Feedback on performance given 


85 


Presented in context 


98 


IIL Instructor 




Direct experience with course material 


63 


Amount of experience with this course 


33 


Previous ratings 


11 
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Program Evaluation Checklist Items 


Percentage of 
Companies 
Supplying 
iniuru luiiui I 


IV. Past Experience 




Number of times course given 


82 


Number of clients 


61 


Number of small businesses 


38 


Adjustments made for small businesses 


72 


Client list available 


63 


V. Cost 




Length of program 


93 


Cost of program 


89 


Discounts available 


74 


VI. Availability 




How soon program can be offered 


91 


When can program be offered 


89 


VII. Evidence of Effectiveness 

1 


56 



We recorded where we were able obtain the information for the program 
evaluaHon checklist. Table 4 shows what source or sources provided the information. 
The table shows the percentage obtained from each source if we were able to obtain it 
all. For example, if we were able to get information about training needs analysis, 70 
percent of the time we got it from the interview only. 

Generally, multiple sources are needed to obtain the information needed for 
course evaluation and selection. If one relies only upon a brochure or only upon an 
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interview, insufficient information will be available. Using both sources greatly improves 
the percentages. 

Different types of information are available from different sources. For example, 
information about course objectives is commonly found in the brochure, but information 
about a training firm's past experience is normally obtained in the interview. 

Table 4 

Percentage Distribution of the Source of Infonnation Needed for 
Course Selection and Evaluation 



Program Evaluation Checklist 


Sources of Information 
(B=brochure I=interview M=malerials) 


Items 


B 

only 


I 

only 


M 

only 


B&I 
only 


B&M 
only 


I&M 

only 


Other 


I. Needs Analysis 
Will do TNA 


20% 


70% 




11% 








II. Course Content 
















Objectives available 


68 


6 


4 


15 






7 


Outline available 


61 


14 


11 


9 


2 


2 




Pretest of competencies 


50 


36 


2 


10 




2 




given 
















Postest of competencies 


43 


40 


3 


10 






4 


given 
















Adjustments made 


39 


50 




11 








based on course 
















participants 
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Program Evaluation Checklist 


Sources of Information 
(B-brochure I=interview M=materials) 


t T fpm Q 


B 

only 


I 

only 


M 

only 


B&I 
only 


B&M 
only 


I&M 
only 


Other 


Methods used 


DO 




4. 


22 


2 


2 


16 1 


Appropriateness of 


41 


23 


11 


14 


2 


4 


c tl 


















Active learning 


34 


25 


11 


16 




9 


5 


1 onnortunities 
















Practice of new skills 


31 


31 


12 


14 




7 


5 


1 Pf>f>HhapW on 




38 


8 


10 




5 


5 


performance given 
















1 Presented in context 


38 


36 


7 


13 





2 


4 


IIL Instructor 
















1 i-'irect expenenct: wuii 


28 


55 




14 






3 


course material 
















Amount of experience 


27 


60 




7 






7 


with this course 
















Previous ratings 




100 













IV. Past Experience 

Number of times course 
given 

Number of clients 

Number of small 
businesses 

Adjustments made for 
small businesses 

Client list available 



27 

36 
8 

12 
14 



64 

57 
92 

85 

86 
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Program Evaluation Checklist 


Sources of Information 
(B=brochure I=interview M=materials) 


Items 


B I M Bid B&M I&M Other 
only only only only only only 


V. Cost 

Length of program 
Cost of program 
Discounts available 


44 51 — 5 _ — — 
51 49 - — - ~ " 
44 56 — — — " 


VI. Availability 

How soon program can 
be offered 

When can program be 
offered 


41 56 ™ 2 


VII. Evidence of Effectiveness 


19 73 8 ~ ~ — 



These results concerning large training companies have been primarily concerned 
with obtaining the required information. We also obtained some results concerning the 
actual services that training companies provide which will be discussed below. 

Before selecting an education or training program a company should decide 
whether anyone actually needs further training by conducting some form of training 
needs analysis gNA). Small businesses frequently do not have this expertise in house 
and may need to rely on ouL^ide help. Many of the large training companies said that 
they could provide a training needs analysis (Table 5). The average time needed for the 
analysis for those willing to conduct one appeared to be one week, indicating a 
rudimentary approach. A quarter of the training firms said that they would be willing to 
assist a small business conduct their own TNA. 
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Table 5 

Statistics Concerning Training Needs Analysis Derived 
from the Survey of Large Training Companies 



Percentage willing to conduct 
aTNA 39% (46) 

Mean time needed to complete 

the TNA 5.5 days (10) 

Percentage willing to help a small 
business to conduct a TNA 24% (29) 



Note: Numbers in parentheses indicate the number of 
programs from which the information was obtained. 



We also collected some information concerning course content (Table 6). Only 
about one-half of the companies listed their training objectives in the brochure or course 
materials. Others had general objectives, but had not clearly identified for 
communication to possible participants. Of those who could identify their objectives, 
most had d.t least some objectives that could be measured. In terms of measuring the 
competencies of the participants before and after the program, a minority of courses 
could do this. A higher percentage assessed competencies after the program than 
assessed it before. Course outlines were available for 83 percent of the courses. A 
majority of courses could be adjusted somewhat depending upon the participants in the 
program. 
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Table 6 

Statistics Concerning Course Content Derived 
from the Survey of Large Training Companies 



Percentage giving training 
obiectivGS 




53% (45) 


Measurement of objectives 






All are measurable 


29% 




Some are measurable 


54 




None are measurable 


17 

100% 
(24) 




Percentage giving course outline 




83% (43) 


Percentage assessing competencies 
before the program 
after the program 


39% (41) 
35% (40) 


Percentage who will adjust course 
based upon the participants 


61% (44) 



The courses offered continued to rely upon the traditional instructional methods, 
with the highest percentage of connpanies using lectures (Table 7). Discussion was the 
second most common method, followed by videos and simulations. Simulations can be a 
very effective tool if they are related closely to required work on the job- 
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Table 7 

Percentage of Latge Training Companies Using Various Training Methods 





Percentage Using this 


Method 


Method m the Program 


Lecture 


67% 


Self-study 


11 


ViHpo or film 
▼ ivi^v/ v/i mill 


35 


Audio 




Interactive video 


9 


Computer-based training 


22 


Discussion 


50 


Role-plays 


17 


Case studies 


26 


Simulations 


37 


Work conferences 


15 


Field projects 


2 



In the previous chapter we discussed the fact that evaluation of training appears 
to be rare, despite its importance. We asked about the types of evaluations that had been 
done for the courses we sampled. The most common type of evaluation performed is 
trainee reaction, which is Kirkpatrick's first level of evaluation C^able 11). Pre- and post- 
testing, changes in behavior, and measurement of effects on the organization are 
relatively rare. Although it is not a form of evaluation, many companies were willing to 
provide selected testimonials from satisfied customers. 
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Table 8 

Types of Evaluations Perfonned by Large Training Companies 





Percentage 
Performing 


Selected testimonials 


54% 


Trainee reactions 


39% 


Pre and post testing 


11% 


Changes in behavior 


9% 


Effect on organization 


9% 



We were only able to collect this extensive information for large nationwide 
training companies. A variety of methods were used to learn about other sources of 
training. 

Small Local Training Vendors 

We contacted local vendors within the metropolitan Washington area. We 
excluded branches of large national companies. One hundred and seven vendors were 
contacted by telephone. More than half did not return our phone calls. Many of these 
companies are small and do not appear to have full-time secretarial support; answering 
machines were very common. Of those we did reach, many offered training programs 
outside the scope of our study. We managed to complete a program evaluation checklist 
for eleven courses. While this is too few to perform a statistical comparison with the 
large nationwide training vendors, we noted certain patterns: 

Brochures from small companies frequently cover all programs, not just 
one course. This makes it more difficult to assess course objectives and 
content. An interview is even more important with the small training 
companies. 
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• When interviewing a small vendor you are usually talking v^ith the 
President v^ho may teach some of the courses. This makes it easier to find 
out about the course. The large nationwide training programs have 
professional sales people who may not be as knowledgeable about training. 

• Small companies tend to offer programs at a lower cost than the larger 
firms. 

• Small companies generally tend to use the same methods as the large 
companies, but appear to use more role plays. This may just be an artifact 
of a small sample, however. 

• Most local vendors appear to be very willing to tailor a program to a small 
firm's needs. 

Community Colleges and Local Universities 

We contacted community colleges and colleges having adult education programs 
in the Washington, D.C. area. In this metropolitan area alone, we found over a dozen 
programs. Community colleges appear to be targeting small businesses and are eager to 
work with them. In our interviews with college personnel, we asked them which courses 
were offered on our list. Almost all courses were offered by every institution. However, 
this does not mean that the course is necessarily being offered as a college course. 
Instead, the community colleges appear to have consultants who offer courses in various 
areas. When someone requests a course they are able to provide it. 

Costs are quite low. Prices quoted are $5 to $15 per student per hour. These 
prices are even less expensive than either national or local training vendors. 

Community colleges are quite willing to tailor a course for a specific firm. 
Consortia 

One of the ways of sharing the costs of training for small businesses is to have 
groups of similar businesses band together to develop the materials. This appears to be 
occurring through the development of consortia. We began with learning of the 
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American Institute of Banking's training program. This allows banks to offer specialized 
courses for their staff without having to develop the course themselves. We contacted 
associations to learn of other programs. Those that we uncovered appear to be a small 
sample of those that are available. For example the National Association of Home 
Builders has a Home Builders Institute that offers courses, such as building codes and 
standards, building technology, computer applications, estimating, to name a few. These 
courses are offered in many states and could be convenient for a small business 
specializing in home building. Other consortia include: National-American Wholesale 
Grocers' Association, National Foundation for Women Business Owners, Association of 
Government Accountants, and the American Institute of Certified Public Accountants. 

Most small business have a national association of businesses similar to their own. 
Many of these associations also have training arms that can provide industiy-specific 
training. This source of training is more important than we realized at the beginning of 
this project. 

B. Study of Small Business Training Practices 

Four different types of focus groups were held with small business owners or 
human resource directors. Three of them were conducted by AIR staff and one was held 
by another organization and a questionnaire was given to participants. For all of the 
groups we collected information about their current training practices, particularly in 
regard to choosing a vendor. 

The National Association of Manufacturers held their own series of focus groups 
with CEOs of small businesses around the country for the Department of Labor. They 
distributed a one-page list of questions about small business training that participants 
could return to AIR if they wished. Eight participants returned the survey. All of these 
respondents purchased some training for their staff from outside vendors. The factors 
most often considered in choosing a vendor were: 

• cost 

• course content 

• previous experience of the training company 

• delivery mode 
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Some respondents also considered training goals and objectives. The most common 
method for locating training vendors was through personal contacts at other companies 
or through local groups, such as the Chamber of Commerce. 

The three focus groups conducted by AIR were located in Raleigh, North 
Carolina; Cincinnati, Ohio; and Herndon, Virginia. At each of these focus groups 
participants were asked about their current training practices. The participants for the 
focus group in Herndon, Virginia were from larger companies than those in Raleigh and 
Cincinnati. This affected the responses of the groups. 

In both the Raleigh and Cincinnati focus groups, respondents reported that they 
do little training. They generally agree that they should do more, but it is difficult to 
spare the time and the money. On-the-job training becomes the most common type of 
training provided and even this is sometimes inadequate. Problems they recognized as 
being created through this lack of training include turnover, equipment breakdowns, 
losses in sales, and increased call backs. 

One reason why training may not be getting the attention it deserves is that only 
one of the companies in Raleigh and Cincinnati had a training person. At all of the other 
companies, training was handled by the CEO or department heads. 

When training is provided, our focus group participants are not likely to compare 
vendors. If a relevant training brochure arrives at the time a particular type of program 
is sought they will consider sending a person to that program, but may not compare it to 
any other program. The decision about whether to use a program is influenced by cost 
and whether the program can be offered at a convenient time. Courses offered through 
community colleges and local groups, such as the Chamber of Commerce, are considered 
to be cost effective. 

The Herndon Virginia focus group had somewhat different results. These 
companies were larger (150 to 420 employees) and the participants at the meeting were 
human resource directors from these firms. Accordingly, these companies are providing 
an array of programs, including basic skills and TQM. 

These human resource directors rely on each other to find good training 
programs. They also use industry-specific associations, national training organizations. 
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and local universities as possible sources. Prior to purchase, programs are reviewed if 
possible. Sometimes a program can be observed at another company. 

We realize that these focus groups cannot provide a full picture of training 
programs used by small businesses. Nevertheless, we gained insights from the groups 
that we incorporated in our initial draft of the guidebook- Some of these ideas were: 

Size of the company seems to make a difference in terms of how training 
decisions are made. Small companies do not usually have anyone who 
specializes in training. 

Even though they may not be doing it, small companies have some 
appreciation of the need for training. 

A small company cannot afford to send many employees to training at the 
same time. 

The relative merits of different training programs are rarely compared. The 
decision is usually whether to do a specific program, rather than 
comparing different sources for the same program. 

Cost is a key factor in choosing a training program. 

If a training program does not announce itself through a brochure, it will 
be relatively unlikely to be chosen. 

Local organizations, such as the Chamber of Commerce or the Council for 
Entrepreneurial Development in Raleigh, provide important opportunities 
for training. In addition, local community colleges are attractive due to 
low cost. 

Word of mouth is very important in finding relevant training programs for 
small businesses. 

Managers and supervisors tend to get more training than workers. 
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Sometimes training is not given to increase productivity, but rather to 
improve morale or as a "perk." 

Training is not usually planned, but rather is given ad hoc, in response to 
events or an attractive opportunity. 

C. Drafting the Guidebook 

The information on training programs and small business practices was presented 
to the Advisory Committee. They had some suggestions for the format of the guidebook 
and sections to include. The consensus was that the guidebook would consist of three 
sections, including: training needs analysis, guidelines for selection, and evaluation. AIR 
research staff drafted sections of the guidebook and AIR's Document Design Center staff 
reformatted it to make it as easy to understand as possible. 
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The guidebook is intended to be useful to small business owners and managers 
who must choose training for their staff. In order to test the guidebook with this 
intended audience, we distributed copies to small businesses throughout the United 
Stales. In this chapter we will describe the results of this usability testing. 

A. Distribution of the Guidebooks 

In order to get some input from small businesses on the guidebook, we mailed a 
guidebook along with a short questionnaire to a variety of respondents. Four methods 
of obtaining respondents were used: 

1. We sent the guidebook to participants in our focus groups. Fourteen 
participants were in focus groups in Virginia, Ohio and North Carolina. 

2. We asked several members of our advisory committee to distribute copies 
of the guidebook to small business persons they know. This accounts for 
another 16 questionnaires distributed in Minneapolis and the District of 
Columbia. 

3. We contacted Small Business Development Centers (SBDCs) in various 
states and asked them to assist us by distribuHng questionnaires to small 
business persons that they know. Usually the state director of the SBDC 
said that he or she would distribute the questionnaires to the regional 
directors within the state or directly to small businesses themselves. This 
method accounts for the majority of the questionnaires distributed-156. 
The states who agreed to participate and the number of questionnaires 
distributed to each were: 
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Number of Questionnaires 



Alabama 

Arizona 

California 

Connecticut 

Delaware 

Illinois 

Iowa 

Montana 

Nebraska 

Oregon 

Pennsylvania 

Rhode Island 

Tennessee 



12 
20 
24 
10 
2 
16 
10 
10 
6 
1 

10 

20 
15 



4. After using the first three methods mentioned above, we ran an ad in The 
Washineton Post looking for small business persons who would like a 
guidebook such as ours in return for completing an evaluation of it From 
this source we distributed 10 guidebooks. 

In total, 196 guidebooks were distributed. From all of these methods, we received 
19 responses, or approximately 10 percent. This is a low rate of return, but probably is 
to be expected given the method of sampling. Our methods of distribution were very 
indirect and we had no leverage over the potential respondents. ParHcularly with the 
SBDCs we had no control over who the guidebooks were given to. Nevertheless, the 
respondents are geographically distributed throughout the country and some patterns 
emerge that are fairly consistent. 
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B. Characteristics of the Respondents 

We had responses from all sections of the country as follows: 



West California 2 

Montana 1 

Oregon 1 

Midwest Minnesota 4 

Nebraska 1 

Ohio 2 

Wisconsin 2 

Northeast District of Columbia 1 

Maryland 1 

Rhode Island 2 

South North Carolina 1 

Alabama 1 



Of the 19 responses, 4 were from heads of Small Business Development Centers. 
Their responses are of interest due to their frequent contact with small business people. 
However, we want to separate these responses from those from actual businesses. From 
this point on we will discuss the results from the small businesses and then make some 
comments about the reactions of the persons from SBDCs at the end of this chapter. 

We asked respondents from some characteristics about their business. The 
businesses represented show a wide range as indicated by years in business, gross 
revenue and number of employees. However, most of the businesses were relatively 
small; half of them had 20 employees or less (Table 10). 

The person responding to our survey was almost always the person who makes 
training decisions within the company. Only two respondents said that another person 
made the training decisions. The position of the survey respondent varied, however. 
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Most commonly the respondent was the president or CEO of the company or the 
director of human resources. 

The number of persons each company had sent to training in the last year varied 
between zero and 25, with an mean number of 6.9. 

Table 10. 

Characteristics of Survey Respondents 



Characteristic 


N 


Low 


Average 


High 


Gross revenue 1992 


10 


$0 


$5.4 million 
(mean) 


$18 million 


Number of 
employees 


14 


1 


20 
(median) 


250 


Years in business 


14 


2 


20 
(median) 


64 



C. Perceptions of the Guidebook by Small Businesses 

We asked respondents questions about their initial reacHon to the guidebook, their 
evaluation of the content and format, and how they used the guidebook to make 
decisions. 

The initial impression of readers when picking up the guidebook was usually 
favorable. On a 7-point scale, the mean response was 5.1. However, a minority of 
respondents had negative reactions. The most common comment was that the cover was 
dull and boring. It appears that a different cover would attract more readers. 

We wanted to know how readers would actually use the guidebook. All but one 
of the respondents read the guidebook at one sitting rather than over a period of Hme. 
Eighty percent spent between 11 and 60 minutes reading the guidebook. We intended 
the book to be easy to read and this goal appears to have been fulfilled. 
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Respondents rated the guidebook on a number of 7-point scales as shown below. 
For all of the scales, except the first, a higher score is a more favorable result We have 
marked the mean score below the scale. Brackets indicate the range of responses. 

The level of detail for these respondents appeared to be "about right" Eight of 
the respondents rated the level of detail as "4." Of those who disagreed, three thought 
the level of detail was slightly too little and three thought it was slightly too much. 

Table 11. 

Reactions of Respondents to the Guidebook 



Level of Detail: 



Too little 
Detail 
1 2 



3 
I- 



About 
Right 

4 5 
"M (14)] 



Too 
Detailed 
6 7 



Organization: 



Writing Clarity: 



Format: 



Disorganized 

1 2 



Confusing 
1 2 



Difficult 
to Read 
1 



3 
l" 



4 

[- 



Well 
Organized 
5 6 7 
5^ (14) ] 

Clear 
5 6 7 
-5^ (14) ] 

Easy 
to Read 
5 6 7 

~~-5A (14) ] 



Examples: 



Poor 
1 



3 
I- 



Excellent 
5 6 7 

-5^(14) ] 
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All respondents felt that the guidebook was organized. Everyone responded at 
the midpoint or higher, yielding a mean of 5.6. A similarly high mean was obtained for 
writing clarity, although one person did rate the guidebook a three on this variable. In 
terms of format, the mean was slightly lower, again with one rating below the midpoint 
on the scale. Examples received the relatively lowest rating, although the mean was still 
above 5.0. 

After completing these specific ratings, respondents were asked to give an overall 
rating in terms of how useful they felt the guidebook would be to them. Three-quarters 
of the respondents gave a 5, 6 or 7 on a 7-point scale, yielding a overall mean of 4.9. 
However, one-quarter of the respondents rated the guidebook as a 1, 2 or 3. 
Unfortunately, the comments of those who gave low ratings were not clear as to why the 
low ratings were given. One individual complained that the book was too simple. 

Rjspondente were asked to describe the most useful information in the guidebook 
and the least useful. Of those who responded, all but two said that the guidelines or 
checklist were the most useful. This is an important result, since the guidelines are the 
critical element in the book. Other useful information mentioned was the section on 
determining training needs and the list of possible sources for ti-aining. 

The comments on the least useful information were varied. Several persons said 
tiiat everything was useful-no wasted information. Other comments were received from 
only one person as follows: 

Why not indicate specific resources, i.e.. Encyclopedia of Associations? 

• More info [needed] on needs analysis 

• How do you begin section 

I looked for more mention of supervisory type training/ reference-think 
there is a great need at this level 

• Examples regarding program choice 
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We also asked respondents which sections need revision. One person suggested 
listing addresses of trainers, videos, books and seminar leaders as a first source to 
locating more. Another respondent suggested revising the "How do you begin" section. 
Another suggested more examples from the small manufacturing sector. 

C. Perceptions of the Guidebook by Small Business Development 
Centers 

When we sent the guidebook to Small Business Development Centers to distribute 
to small business people, a few of them reviewed the guidebook and sent a questionnaire 
back to us. Of the four that did so, three were critical of the guidebook and one liked it 
very much. For instance, three of the four rated the usefulness of the guidebook below 4 
on the 7-point scale. Those who were critical appeared to have the same reacHon. The 
guidebook is too basic for them and they do not think it will be useful. One respondent 
feared that the guidebook might not be reliable or specific enough. Another wrote that 
the guidebook, "was designed for third grade reading." He went on to say, "As one who 
offers training your guide was so basic it was insulting to me as a reader. The pages 
have few words, great white space and no depth of analysis." 

D. Discussion 

The reaction to the guidebook was not uniform. Some respondents liked it very 
much and felt it would be very useful to them. Examples of these reactions are as 
follows: 

This is basically the process we use, but [it is] very nice to have in this 
booklet form. 

This guidebook is terrific! 1 wish we had it earlier this year. As it was, I 
had to do all of this work on my own through trial and error . . . Every 
training director and owner of a small business should get a copy of this. 

I intend to use this guidebook in developing training materials for start-up 
microbusinesses. While not exactly the use it was designed for, the book 
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provides a useful framework to me in clarifying how such training could 
best be structured. 

I like the piece. It is something I would recommend or give to people in 
small and mid-sized businesses with whom I work. For its intended 
audience, it is the right combination of comprehensive but easy to use. 
There is a real tendency for those of us in the training/O.D. business to 
insist on saying too much, or cramming so much into a guidebook that it 
becomes hard to use or a turn off. You've hit the desired middle ground. 

Others felt the guidebook was too simplistic, as reflected in some of the comments 
by the SBDC respondents and a few of the small businesses. It seems safe to conclude 
that this guidebook will not please everyone. For those who have little training 
background the book may give them just what they need. As they gain in sophistication 
they can go on to other sources of training information. For more experienced people, 
some will find the guidebook useful as a structure for decision-making or review. 
However, others may find it too basic. 
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Chapter 6. Discussion 



This project has been concerned with helping small businesses improve their 
approach to training. The effort was timely in that it responded to a growing national 
recognition of the importance of small business to the American economy. The 
guidebook that was developed was specially designed to fit the capabilities and 
operational constraints of small businesses and was aimed at helping them compete in 
the global markets through better training practices. 

The project was a departure from the mainstream of current research because it 
merged the interests of both the educational and training communities. The Department 
of Education funded the project, but many of the issues are traditional concerns of the 
Department of Labor. A fundamental assumption of the project is that business pracHce 
can be improved through education and training of the work force. The project 
demonsrates that training and education are on the same continuum and share the same 
fundamental concerns and goals - human learning. 

During the course of the project, AIR's research team learned a great deal about 
the needs and concerns of small business, especially as they relate to ways of improving 
the abilities and skills of the work force. More interestingly, perhaps, our work 
uncovered many other aspects about small businesses that are not well understood and 
deserve careful examination and research. 

One issue that surfaced and was prevalent throughout the project is simply, what 
is a small business? We operated on the Federal definition that firms that emplo> less 
than 500 employees are "small business." Whatever value this categorization provides 
other government funcHons and interests, it is not particularly helpful in understanding 
training requirements and designing optimal training strategies. The definiHon of 500 
employees or less covers a wide range of company resources and capabilities. A 
business with 400 employees that has a human resources department and training 
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specialists on staff is very different from a firm with 20 employees with no training 
expertise in any position. 

Smaller businesses appear to be primarily occupied with survival. Training is 
frequently seen as a cost rather than an investment. Presidents or managers who take on 
the training function are busy tending to bottom line issues such as profit. While some 
of these managers may see training as desirable but unattainable, many others do not 
view it as important or as a way of improving productivity. Even those who provide 
training may do so to improve employee morale or other reasons unrelated to job 
performance. Company size thus seems to be a variable of significance in planning 
training. That is, the need for training and how to design and deliver needed training 
will vary as a function of company size. Our work on this project did not allow us to 
gain definitive insight into this issue, but it appears to be a worthwhile area of research 
that would contribute to finding ways of helping small business improve through 
training and education. 

Current social trends also are influencing the training needs of small businesses 
just as they are for large business, but small businesses usually have less capability to 
understand the implications of these trends and to take appropriate action. The increase 
of immigrants, non-English speakers, and minorities in the labor force has profound 
influence on training needs. The downsizing of large companies is bringing a different 
mix of job seeker with different expectations and skills than many small firms commonly 
see. This, too, has training implications. The decline of manufacturing opportunities and 
concurrent increase in service-oriented business also bring different training needs to the 
workplace. The influx of new and rapidly changing technology affects how employees 
do their job in small companies as well as lai^e corporations. The impact of trends such 
as these on small business training needs is another area of fruitful research. 

In addition to what we have learned about small business, we have also some 
information about the training industry. The information needed to make an informed 
decision about a training program is difficult to obtain. Getting answers requires time 
and persistence that few businesses may have, even if they are large. The large national 
training companies employ sales personnel who are frequently more interested in sellmg 
than in meeting anyone's needs. Evaluation of training programs that are being 
marketed seem to be very rare, with the exception of trainee reactions. Programs are 
sold and used with little idea of whether they work or not. The need for a guidebook 
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such as the one developed in this project serves a very important function of trying to 
make purchasers of training and education programs more informed consumers. 

Yet, certain training sources appear to be more critical in terms of any eventual 
solution to the problem than we realized as we began the project Local organizations of 
small businesses, such as the Council for Entrepreneurial Development in North 
Carolina, are offering help that is relevant and useful. In addition, the community 
colleges appear to be attuned to the small business problem. They offer more courses 
than small businesses actually use. But even in these cases, the small business training 
user still should be able to judge the quality of training services or programs these 
sources offer. 

More research should be done to understand more fully the problem of education 
and training in small businesses. Some companies are doing much more than others. 
What differentiates the company that does more relevant training from the one that does 
nothing? Is it strictly a function of size of the company? Perhaps the companies that 
really need help are those below a certain critical size. On the other hand, is the 
presence of a training professional the key factor? 

For firms who are training some employees, what kinds of programs are they 
using most frequently? What programs appear to be most successful in terms of small 
business needs? 

This is an area in which government help may have an impact. Before embarking 
on any program, however, one should be clear on what kinds of intervenHons are likely 
to be useful. Is cost the critical barrier to training in small businesses? If the cost of 
courses were subsidized, would more companies take advantage of them? Would it be 
more effective to put the money into the hands of individuals? If individuals had 
incentives or subsidies for training would they be more likely to take the initiative for 
their own development? 

Even more fundamentally, how much of a benefit does a small firm get from 
training its employees? Do employees who are trained leave or do they stay and feel 
some loyalty to an employer who develops their potential? If the value of training could 
be demonstrated to small businesses would they be more likely to use it? 
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The research agenda for probing the training and education needs of small 
business (and perhaps larger businesses as well) from the Department of Education 
perspecHve, is both broad and potentially rich. However, one of the lessons learned 
from this project is that the Department of EducaHon is a largely foreign enHty to the 
business community; it is not seen as a natural source of wisdom and help. Business 
managers and execuHves are familiar with and attend to iniHatives, direcHves, and 
regulations issued by the Federal Department of Ubor, Department of Commerce, 
Internal Revenue Service, Small Business AdministraHon, etc., but rarely do they 
encounter the Federal agency that "oversees" their children's school. This is important 
because to gain the attention of small business and to enlist the cooperation of individual 
business firms in educaHonally-based research and development, support from business 
associaHons and groups for the research should be part of the research design. That is, 
support from such organizations as the National Federation of Independent Business, 
U.S. Chamber of Commerce, and the National Alliance of Business, either in the form of 
endorsements or as co-members of the research team, would facilitate access to the small 
business community for purposes of conducting training and educational research in the 
workplace. 

In summary, we believe that our guidebook is a useful product for the small 
busin-ss person who is not a training professional. To use the guidebook, however, the 
firm needs to be actually seeking training for its staff. Our guidebook will not be of 
interest to a business that has no intention of providing training. DistribuHon of the 
guidebook through the right channels becomes even more important when faced with 
this reality. The next chapter discusses dissemination options. 
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We investigated many options for disseminating tlie guidebook. We are most 
concerned tliat the guidebook reacli its intended audience-small businesses. At the time 
of this writing a definite plan has not been made, but we do have possibilities. 
Organizations contacted include: 

1. National Alliance of Business 

The National Alliance of Business publishes books and pamphlets and then sells 
them to members. Generally the NAB works with larger businesses. We sent a draft of 
the guidebook to Brenda Bell, Vice-President of Marketing, for review, but they were not 
interested in publishing the document through their own distribution network. They still 
may be willing to publicize the guidebook through their newsletter. 

2. Small Business Administration 

We have had contact with George Solomon, who works with business education 
at the Small Business Administration. The SBA has an on-line information service that 
he thinks could possibly be used for disseminating the information in the guidebook. 
This would lose the advantage gained by our formatting the document so that it is easy 
to read, however. After publishing the document in hard copy, it may be worth 
considering whether an on-line version would be useful. 

3. National Federation of Independent Business 

The NFIB never expressed any interest in our project. It is possible that the right 
people were never reached. 
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4. American Society for Training and Development 

ASTD publishes a series of small guidebooks on various training topics called 
Info-Line. Members can subscribe to the series or buy individual copies. The price is 
$10, regardless of length. 

We spoke to the editor of Info-line, Barbara Darrell, and also Nancy Olson, about 
our project. Publication through them is a possibility, but the initial review of the 
document by training professionals was not that positive. They would want substantial 
changes and reconsideration by the committee to publish it 

ASTD is a well-known and respected training organizaHon. However, the 
audience for our guidebook is not training professionals, but rather people who do not 
have any training background. If the guidebook were published through ASTD it should 
be distributed through other channels as well. 

5. National Association of Small Business Development Centers 

In major ciHes throughout the country Small Business Development Centers help 
new businesses get started and provide help and information for small businesses that 
are more established. They do not publish materials, but they do distribute them. 

Regardless of publisher, the guidebook could be advertised at a booth at their 
annual convenHon in the fall. This could be an excellent way to have the center directors 
become aware of the guidebook. 

6. National Center for Manufacturing Sciences 

The NaHonal Center for Manufacturing Sciences expressed early interest in the 
guidebook. They would like to distribute it through their 25 centers throughout the 
United States. This would reach the manufacturing sector, but not other sectors. A co- 
publishing arrangement with another company, such as Lakewood, might be a better 
option. 
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The puiMications person at NCMS is Hilary Handwerger, who can be reached at 
(313) 995-0300. NCMS seems to be committed to publishing the guidebook in some 
form, but some details about the arrangements remain. 

7. Lakewood Publications 

Lakewood publishes Training magazine, as well as other publications in the 
training field. They are interested in possibly publishing the document if another 
organization that were closer to small business could help with distribution. Our contact 
there is Linda Klemstein at (612) 340-4848. 

Currently they think they would like to broaden the audience for the guidebook 
beyond small businesses and include other material with the guidebook. They are 
comfortable with the concept that NCMS might publish the document in one way and 
they would do it another way. 

8. U.S. Chamber of Commerce 

A copy of the guidebook was sent to Raye Nelson at the Center for Workforce 
Preparation. Initially she was interested in possibly publishing the book, but has not 
been in contact with us recently. They do not appear to be enthusiastic to publish the 
guidebook. 
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Program Evaluation Checklist 



Name of Program, 
Name of Vendor^ 
Rater's Initials 



Type of Program (Choose only one from this list) 

Management Skills/Development □ 

Delegation Skills □ 
Decision-making ^ 
Managing Change □ 
Supervisory Skills D 

Motivation ^ 
Problem Solving ^ 
Negotiating Skills □ 
Communication Skills O 

Interpersonal Skills □ 
Team Building D 
Listening Skills CD 
Writing Skills □ 
Basic Computer Skills □ 

Word Processing ^ 
Personal Computer Applications □ 
Computer Programming □ 
Data Processing CD 
Management Information Systems □ 
Quality Improvement □ 
Customer Relations/Services 
Executive Development 

Leadership 
Clerical/Secretarial Skills 
Personal Growth 

Time Management 
Goal Setting 
Stress Management 



□ 




□ 






□ 


□ 




□ 






□ 




□ 




□ 



Which sources of information were available? (check all that apply) 

□ Brochure 

□ Interview Position of person interviewed 

□ Examination of Course Materials 

□ Observation of Course (or portions of course) 
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For all of the following questions, indicate the source of the information by supplying a letter in front of each question 
using the following codes: 

B— Brochure 

I-Interview with vendor representative 
M-Examination of course materials 
O-Observation of course 



1. 



Needs Analysis 

Is the vendor willing to do a training needs analysis? □ Yes □ No 

that there are 10 persons who possibly need to be trained in whatever the course is under 



If yes, assume 
consideration. 



How much time would be needed to complete the analysis? days 

How much would such a TNA cost? $ 



What methods does the vendor propose for completing the TNA? 



Is the vendor willing to guide a small business in completing a training needs analysis? □ Yes □ No 



II. Course Content 

Are training objectives listed for the program? □ Yes □ No 

Are these objectives measurable in behavioral terms? 

□ All arc measurable 

□ Some are measurable 

□ None are measurable 

Is a course outline available? □ Yes □ No 

If yes, are the subjects arranged in a logical sequence? 
Comments 



□ Yes 



□ No 



.\re participant's competencies assessed before the program begins? □ Yes □ No 
If yes, please explain 



Are participanfs competencies assessed after the program is completed? □ Yes □ No 
If yes, please explain 



Are adjustments in the course possible depending upon the backgrounds of the participants? 
□ Yes □ No If yes, explain 
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What methods will be used in the training? (check all that apply) 



□ 


Lecture 


□ 


Self-study 


□ 


Video or Film 


1—1 

LJ 


Audios 


□ 


Interactive Video 


n 


Computer-based Training 


□ 


Discussion 


n 


Role-plays 


□ 


Case Studies 


n 


Simulations 


n 


Work Conferences 


n 


Field Projects 



How appropriate do these methods appear to be for the subject matter involved? 



What opportunities are there for active learning? 



What opportunities are there to practice new skills during the training? 



During these practice periods, how is feedback on performance given? 



Does the course emphasize the context of the job/company of the participants rather than just prescming material 
without a context? 



□ Yes □ No 
If yes, desCTibe_ 



Does the instructor have direct experience with the subject matter being taught? 
□ Yes □ No 

If yes, describe 
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How long has the instructor been leaching this course? 



□ 
□ 
□ 
□ 
□ 



Has never taught it 

I to 5 offerings 
6 to 10 offerings 

II to 15 offerings 
16 offerings or more 



Are ratings of the instructor available? 



□ Yes □ No 



If yes, please describe 

Past Experience 

Number of times program has been given 

Number of different companies for whom the program has been given 

Number of small businesses (< 500 employees) that have used the program 

Are any adjustments made for small businesses? □ Yes □ No 
If yes, what are they? 

Is vendor willing to supply names of clients for whom program has been run? 
□ Yes □ No 

Cost 

Length of program (fill in only one) ^hours ^days ^wc 

Program cost per participant $ 

Are discounts available? □ Yes □ No 

If yes, what kind of discount?^ 

Additional costs 

Facility fee $ 

Materials $ 

Equipment $ 

Other $ 
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Availability 



Assume that 10 employees will attend this program, how soon could the course be offered. 

□ Less than 1 week 

□ 1 to 5 weeks 

□ 6 to 10 weeks 

□ 11 to 15 weeks 

□ More than 15 weeks 

When can the program be offered? (check all that apply) 

□ Monday through Friday during normal work hours 

□ Evenings 

□ Weekends 

□ Other - 

Evidence of Effectiveness 

What evidence of effectiveness is available? (check all that apply) 

□ Selected testimonials from users 

□ Trainee reactions to the program 

□ Assessment of participant performance and knowledge by tests (pre-test, post-test, etc.) 

Describe ■ 

□ Long-term validation study showing changes in behavior on the job over time 

Describe . 

□ Long-term validation study showing the effect of the training on organizational results 

Describe^ 

□ Other : 
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